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DETAILED ACTION 

1. Claims 1-72 are pending and being examined. Claims 1,18, 35, 47 and 60 are 
independent claims. 

Double Patenting 

2. The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. A nonstatutory 
obviousness-type double patenting rejection is appropriate where the conflicting claims 
are not identical, but at least one examined application claim is not patentably distinct 
from the reference claim(s) because the examined application claim is either anticipated 
by, or would have been obvious over, the reference claim(s). See, e.g.. In re Berg, 140 
F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 
USPQ2d 2010 (Fed. Cir. 1993); In re Long!, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 
1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 
F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 
USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1 .321 (c) or 1 .321 (d) 
may be used to overcome an actual or provisional rejection based on a nonstatutory 
double patenting ground provided the conflicting application or patent either is shown to 
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be commonly owned with this application, or claims an invention made as a result of 
activities undertaken within the scope of a joint research agreement. 

Effective January 1 , 1994, a registered attorney or agent of record may sign a 
terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 

3. Claim 35 is rejected on the grounds of nonstatutory obviousness-type double 
patenting as being unpatentable over Claim 1, in co-pending application 12/482,864. 

Although the conflicting claims are not identical, the respective claims between the 
instant application and 12/482,864 describe the same invention. Accordingly, claim 35 
of the instant application is rejected under this obviousness-type double patenting 
rationale. 



Instant application 10/535,864 


Reference application 12/482,864 


Claim 35, User interaction system, 
comprising: a first electrical apparatus; a 
portable pointing device operable by a 
user for pointing to a region in space; 
a camera ) connected to the pointing 
device so that in operation it images the 
region pointed to for taking a picture; 
a digital signal processor, capable of 


Claim 1, User interaction system, 
comprising: 

an electrical apparatus (1 1 0); a portable 
pointing device (101 ,300) operable by a 
user for pointing to a region in space; 
a camera (102) taking a picture; and 
a digital signal processor (120), capable of 
receiving and processing the picture, and 
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receiving and processing data of said 


capable of transmitting user interface 


picture, and capable of transnnitting user 


information (1) derived from the picture to 


interface information derived on the basis 


the electrical apparatus (110), wherein the 


of said picture data to the first electrical 


camera (102) is connected to the pointing 


apparatus; and 


device (101 ,300) so that in operation it 


an object displaying a characteristic 


images the region pointed to characterized 


pattern to facilitate recognition of said 


in that the user interaction system further 


object by the digital signal processor. 


comprises a pattern creation means (116) 




to exhibit a characteristic pattern that is 




created for identification purpose by the 




pointing device of the pattern creation 




means associated apparatus. 



Claim Rejections - 35 USC §112 

4. The following is a quotation of the second paragraph of 35 U.S. C. 112: 

The specification sliall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 
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Claim 6-8, 34-35, 37-38 and 61 are rejected under 35 U.S.C. 112, second 
paragrapli, as being indefinite for failing to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. 

As to claims 6-8 . the claims use: symbols "{}", "-" and It is not clear what these 
symbols means. 

As to claim 34 . the claim states "Electrical apparatus for use in a user interaction system 
as claimed in claim 18, characterized in that interface means are comprised which allow 
the electrical apparatus to send information about supported commands to a pointing 
device as claimed in claim 1 , based on at least one call of the pointing device to the 
electrical apparatus." The claim is generally narrative and indefinite, failing to conform 
with current U.S. practice. They appear to be a literal translation into English from a 
foreign document and are replete with grammatical and idiomatic errors. 

As to claim 35 . the claim states the limitation: "an object displaying a characteristic 

pattern to facilitate recognition of said object by the digital signal processor", in lines 9- 
10. It is not clear whether "said object" refers to the first "object" or not. If it is, it is not 
clear what the limitation means. For the purpose of examination, they are irrelative. 
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As to claims 37-38 . the claims state: "the second electrical apparatus". There is 
insufficient antecedent basis for the limitation in the claims. For the purpose of 
examination, the claims are dependent of claim 36. 

As to claim 38 . the claims state: "User interaction system as claimed in claim 36 in 
which the second electrical apparatus is in fact the first electrical apparatus." It is not 
clear what the limitation mean. 

As to claim 61 . the claim states: "wherein the digital signal processor (120) is further 
arranged to recognize the direction of the pointer relative to the localization beacon." 
There is insufficient antecedent basis for "the pointer" in the claims. For the purpose of 
examination, "the direction of the pointer" is interpreted as "the direction of motion". 

Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 1 22(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 
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Claims 1-3, 10-12, 15, 17, 27-29, 32, 34-41, 44, 46-49, 52-54, 57, 59-60, 68-69 
and 71-72 are rejected under 35 U.S.C. 102(e), as being anticipated by Zon (US 
PUGPub 2003/0156973). 

As to claim 1, Zon discloses a user interaction system (the universal remote 
control system, see flg.3 and [0027] lines 1-7), comprising: 
an electrical apparatus (the monitor devices, see 200 in fig.3 and [0027] lines 6-7); 
a portable pointing device operable by a user for pointing to a region in space (the 
universal remote control system, see fig.3 and [0027] lines 1-7); 
a camera taking a picture (the video camera, see 100 in fig 3 and [0027] lines 6-7); 
and 

a digital signal processor (the data analyzer, see 334 in fig.3), capable of receiving 
and processing the picture (analyzing image, see [0033] lines 1-4), and capable of 
transmitting user interface information derived from the picture to the electrical 
apparatus (using the communication channels and sending control signal to the 
devices, see 350, 360 in fig.3 and [0028] lines 1-8), wherein the camera is connected 
to the pointing device so that in operation it images the region pointed to (see [0027]). 

As to claims 2, 68, Zon further discloses wherein the user interface information 
includes apparatus control data for controlling operation of the electrical apparatus 
(permitting signals and data generated by the camera to be applied to control the 
monitors, see [0028] lines 1-8). 
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As to claims 3, 69, Zon further discloses wlierein tine digital signal processor 
has an object characterizing means for characterizing an object or part of the object 
present in the picture of the region imaged by the camera, by providing first object 
characterizing features to object identification means for identifying the object 
(analyzing/comparing the data generated by one of the monitors to determine a 
monitor specific set of correction commands, see [0033] lines 8-11), and which 
object identification means is capable of outputting object identification data from which 
the user interface information is constructed (generating index data in a LUT for 
retrieving correction commands, see [0033] lines 14-24). 

As to claims 10, 27, 39, 52, Zon further disclose, comprising a camera and 
being capable of sending a picture to a digital signal processor (the video camera, see 
100 in fig 3 and [0027] lines 6-7). 

As to claims 11, 28, 40, 53, Zon further disclose, wherein the pointing device is 
capable of sending a picture to the digital signal processor (sending image from 
camera 100 to processor 330, see fig.3), which is capable of sending user interface 
information to an electrical apparatus based on the picture (using the communication 
channels and sending control signal to the devices, see 350, 360 in fig.3 and 
[0028] lines 1-8). 
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As to claims 12, 29, 41 and 54, Zon further discloses wlierein tlie digital signal 
processor (120) is comprised in the pointing device (the image analysis unit, see 334 
in fig.3 and [0029] lines 11-14 and [0033] lines 8-11). 

As to claims 15, 32, 44, 57 and 71, Zon further discloses, comprising a 
programmable user interface code generator and a wireless transmitter for transmitting 
the code to the electrical apparatus (the programmable ROM, see [0029] lines 14-18). 

As to claims 17, 34, 46, 59 and 72, Zon discloses the Electrical apparatus for 
use in a user interaction system as claimed in claim 1, comprising means which allow 
the electrical apparatus to send information about supported commands to a pointing 
device, based on an a call of the pointing device to the electrical apparatus (display 
menu command, see [0031] lines 13-17). 

As to claim 35, Zon discloses a User interaction system (the universal remote 
control system, see fig.3 and [0027] lines 1-7), comprising: a first electrical apparatus 
(the monitor 200a, see 200a in fig.3 and [0027] lines 6-7); a portable pointing device 
operable by a user for pointing to a region in space (the universal remote control 
system, see fig.3 and [0027] lines 1-7); 

a camera connected to the pointing device so that in operation it images the region 
pointed to for taking a picture (the video camera, see 100 in fig 3 and [0027] lines 6- 
7); 
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a digital signal processor (the data analyzer, see 334 in fig.3), capable of receiving 
and processing data of said picture, and capable of transmitting user interface 
information derived on the basis of said picture data to the first electrical apparatus 
(using the communication channels and sending control signal to the devices, 
see 350, 360 in fig.3 and [0028] lines 1-8); and 

an object displaying a characteristic pattern to facilitate recognition of said object by the 
digital signal processor (the monitor 200b, see 200a in fig.3 and [0027] lines 6-7). 

As to claim 36, Zon discloses the User interaction system as claimed in claim 
35, wherein the object displaying a characteristic pattern is a second electrical 
apparatus (the monitor 200b, see 200a in fig.3 and [0027] lines 6-7). 

As to claims 37-38, Zon discloses the User interaction system as claimed in 
claim 36, wherein the second electrical apparatus comprises a display (the monitor 
200b, see 200a in fig.3 and [0027] lines 6-7). 

As to claim 47, Zon discloses a User interaction system (the universal remote 
control system, see fig.3 and [0027] lines 1-7), comprising: an electrical apparatus 
(the monitor 200a, see 200a in fig.3 and [0027] lines 6-7); a portable pointing device 
operable by a user for pointing to a region in space (the universal remote control 
system, see fig.3 and [0027] lines 1-7); 

a camera connected to the pointing device so that in operation it images the region 
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pointed to for taking a picture (the video camera, see 100 in fig 3 and [0027] lines 6- 
7); 

a digital signal processor (the data analyzer, see 334 in fig.3), capable of receiving 
and processing data of said picture, and capable of transmitting user interface 
information derived on the basis of said picture data to the electrical apparatus (using 
the communication channels and sending control signal to the devices, see 350, 
360 in fig.3 and [0028] lines 1-8); and 

wherein said user interface information includes a specification of the electrical 
apparatus, or of a part of the electrical apparatus intended to be used by the user (the 
display menu commands which is corresponding to the aimed device, see [0031] 
lines 13-17). 

As to claim 48, Zon discloses the User interaction system as claimed in claim 

47, wherein the electrical apparatus, or the intended part of the electrical apparatus is 
identified by determining the position of the electrical apparatus in the picture of the 
region pointed to (the camera located to take the entire image of the device, see 
[0032] lines 5-8). 

As to claim 49, Zon discloses the User interaction system as claimed in claim 

48, wherein the electrical apparatus, or the intended part of the electrical apparatus 
corresponds to a fixed position in the picture (the camera located to take the entire 
image of the device, see [0032] lines 5-8). 
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As to claim 60, Zon discloses a User interaction system (the universal remote 
control system, see fig.3 and [0027] lines 1-7), comprising: 
an electrical apparatus (the monitor 200a, see 200a in fig.3 and [0027] lines 6-7); a 

portable pointing device operable by a user for pointing to a region in space; a camera 
taking a picture (the universal remote control system, see fig.3 and [0027] lines 1- 
7); and 

a digital signal processor (the data analyzer, see 334 in fig.3), capable of receiving 

and processing tine picture, and capable of transmitting user interface information 
derived from the picture to the electrical apparatus (using the communication 
channels and sending control signal to the devices, see 350, 360 in fig.3 and 
[0028] lines 1-8), wherein the camera is connected to the pointing device so that in 
operation it images the region pointed to characterized in that the system further 
comprises at least one localization beacon that can emit electromagnetic radiation, 
which can be captured by the system and for use to the digital signal processor in order 
to recognize to where the pointing device is pointing by using information derived from 
the captured electromagnetic radiation (analog/digital signal generated by the video 
camera, see [0032] lines 5-10). 



Claim Rejections - 35 USC § 103 
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6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



Claims 4, 9, 13-14, 16, 18-26, 30-31, 33, 42-43, 45, 50-51, 55-56, 58, 61-67 and 
70 are rejected under 35 U.S.C. 103(a), as being unpatentable over Zon in view of 
Silverstein (EP 1130906). Silverstein is cited by applicant in IDS and filed on 
12/14/2005. 

As to claims 4, 19, 23, 65, Zon does not disclose motion trajectory estimation 
means for estimating a motion trajectory of the pointing device. However, Silverstein 
does teach a digital signal camera to be used as a pointing device, comprising: 
motion trajectory estimation means for estimating a motion trajectory of the pointing 
device and outputting a first motion characterizing signature, a signature being a 
mathematical abstraction of the motion trajectory (the motion detection unit, see 130 
In fig.1 and [0013] lines 4-6); and 

signature identification means for identifying the first motion characterizing signature 
and outputting command identification data, which represents a user interaction 
command, corresponding with the first motion characterizing signature, from which 
command identification data the user interface information is constructed (the target 



Application/Control Number: 10/535,464 Page 14 

Art Unit: 2624 

icon selection unit whicli is corresponding to the movement of the point device, 
see [0016] lines 16-22). 

It would have been obvious to person skilled in the art at the time of the invention to 
combine the teaching of Silverstein into the teaching of Zon in order to robustly and 
accurately permitting cursor control and designation in digital camera display 
(Silverstein, [0006]). 

As to claims 9, 20, 63, the combination of Zon and Silverstein further discloses 
the first motion characterizing signature is derived on the basis of successive pictures 
imaged by the camera at respective instances of time (Silverstein, using the changes 
between subsequent images to calculate a mount and direction of the motion, see 
[0012] lines 39-41). 

As to claims 13, 21, 30, 42, 55, 62, Zon discloses the Pointing device as 
claimed in claim 10. Zon does not disclose comprising motion sensing means for 
sensing a motion trajectory of the pointing device. However, Silverstein does teach 
comprising motion sensing means for sensing a motion trajectory of the pointing device 
(motion sensor such as gyroscopes, see [0011] lines 17-19, and [0012] lines 41- 
50). It would have been obvious to person skilled in the art at the time of the invention to 
combine the teaching of Silverstein into the teaching of Zon in order to robustly and 
accurately permitting cursor control and designation in digital camera display 
(Silverstein, [0006]). 
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As to claims 14, 31, 43, 56 and 70, Zon discloses tine Pointing device as 
claimed in claim 10. Zon does not disclose comprising a characteristic projector for 
optically projecting a characteristic pattern towards a region pointed to. However, 
Silverstein does teach comprising a characteristic projector for optically projecting a 
characteristic pattern towards a region pointed to (viewfinder, see 160 in fig.1 and 
[0017]). It would have been obvious to person skilled in the art at the time of the 
invention to combine the teaching of Silverstein into the teaching of Zon in order to 
robustly and accurately permitting cursor control and designation in remote control 
device (Silverstein, [0006]). 

As to claims 16, 25-26, 33, 45, 50-51, 58, 66-67, Zon discloses the Pointing 

device as claimed in claim 10. Zon does not disclose comprising feedback means for 
feedback of user interface information. However, Silverstein does teach comprising 
feedback means for feedback of user interface information (displaying view field 
selected by the viewfinder, see [0032] lines 24-27). It would have been obvious to 

person skilled in the art at the time of the invention to combine the teaching of 
Silverstein into the teaching of Zon in order to robustly and accurately permitting cursor 
control and designation in digital camera display (Silverstein, [0006]). 

As to claim 18, Zon discloses a User interaction system, comprising: an 
electrical apparatus (the monitor devices, see 200 in fig.3 and [0027] lines 6-7); a 
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portable pointing device operable by a user for pointing to a region in space (the 
universal remote control system, see fig.3 and [0027] lines 1-7); 
a camera connected to the pointing device so that in operation it images the region 
pointed to for taking a picture (the video camera, see 100 in fig 3 and [0027] lines 6- 
7); and a digital signal processor (the data analyzer, see 334 in fig.3), capable of 
receiving and processing data of said picture, and capable of transmitting user interface 
information derived on the basis of said picture data to the electrical apparatus (using 
the communication channels and sending control signal to the devices, see 350, 
360 in fig.3, [0028] lines 1-8, and [0027]). Zon does not disclose motion sensing 
means for estimating the motion of the pointing device. However, Silverstein does teach 
motion sensing means for estimating the motion of the pointing device (the motion 
detection unit, see 130 in fig.1 and [0013] lines 4-6). It would have been obvious to 
person skilled in the art at the time of the invention to combine the teaching of 
Silverstein into the teaching of Zon in order to robustly and accurately permitting cursor 
control and designation in digital camera display (Silverstein, [0006]). 

As to claims 22, 61 and 64, the combination of Zon and Silverstein discloses 
the User interaction system as claimed in claim 18 wherein the transmitted user 
interface information includes at least one feature selected from the group consisting of 
motion speed, motion direction, and acceleration of the pointing device (Silverstein, 
calculating direction of motion, see [0012] lines 39-41). 
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As to claim 24, the combination of Zon and Silverstein discloses the User 
interaction system as claimed in claim 18, further comprising room localization beacons 
for emitting electromagnetic radiation, wherein the digital signal processor is arranged to 
recognize to which part of the room the pointing device is pointing (analog/digital 
signal generated by the video camera, see [0032] lines 5-10). 

Allowable Subject Matter 

7. Claims 5-8 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

8. The following is a statement of reasons for the indication of allowable subject 
matter: 

As to claims 5-6. no prior art in record teaches: "wherein the digital signal processor 
includes identification improvement means, which are capable of further improving a 
probability that the object represented as object identification data, and user interaction 
command represented as command identification data, are more reliably identified 
based on predetermined rules, yielding more reliable user interface information". 



As to claim 7 . no prior art in record teaches: "wherein the digital signal processor 
includes object association means for providing to the object identification means object 
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association data comprising at least one of the data entities being: associated object 
characterizing features and object related data, the object association data being 
derivable from object template data in object memory originating from at least one of the 
methods: 

the object template data Is obtained from object training means performing a 
predetermined calculation on second object characterizing features outputted by object 
characterizing means; and the object template data is derived from inputted object 
data." 

As to claim 8 . no prior art in record teaches: "wherein the digital signal processor 
includes signature association means for providing to the signature identification means 
signature association data comprising at least one of the data entities being: associated 
signature features and command related data, the signature association data being 
derivable from signature template data in signature memory originating from at least 
one of the methods: 

the signature template data is obtained from signature training means performing a 
predetermined calculation on a second motion characterizing signature outputted by the 
motion trajectory estimating means ; and 

the command template data is derived from inputted command data." 
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Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. The references cited are listed in the "Notice of Reference 
Cited". 

1 0. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to RUIPING LI whose telephone number is (571)270-3376. 
The examiner can normally be reached on 8:30am-5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Samir Ahmed can be reached on 571-272-7413. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/RUIPING LI/ 
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Supervisory Patent Examiner, Art Unit 2624 



